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Abstract 


Although interaction is recognised as a key element for learning, its incorporation in online 
courses can be challenging. The interaction equivalency theorem provides guidelines: Meaningful 
learning can be supported as long as one of three types of interactions (learner-content, learner- 
teacher and learner-learner) is present at a high level. This study sought to apply this theorem to 
the corporate sector, and to expand it to include other indicators of course effectiveness: 
satisfaction, knowledge transfer, business results and return on expectations. A large Mexican 
organisation participated in this research, with 146 learners, 30 teachers and 3 academic 
assistants. Three versions of an online course were designed, each emphasising a different type of 
interaction. Data were collected through surveys, exams, observations, activity logs, think aloud 
protocols and sales records. All course versions yielded high levels of effectiveness, in terms of 
satisfaction, learning and return on expectations. Yet, course design did not dictate the types of 
interactions in which students engaged within the courses. Findings suggest that the interaction 
equivalency theorem can be reformulated as follows: In corporate settings, an online course can 
be effective in terms of satisfaction, learning, knowledge transfer, business results and return on 
expectations, as long as (a) at least one of three types of interaction (learner-content, learner- 
teacher or learner-learner) features prominently in the design of the course, and (b) course 
delivery is consistent with the chosen type of interaction. Focusing on only one type of interaction 
carries a high risk of confusion, disengagement or missed learning opportunities, which can be 
managed by incorporating other forms of interactions. 

Keywords: Interaction; course effectiveness; online courses; interaction equivalency theorem 


298 



Expanding the Interaction Equivalency Theorem 
Padilla Rodriguez and Armellini 

Introduction 

Interactions with the content, peers or the teacher have long been recognised as an essential 
component of any course. Their value in web-based courses has been reported by learners 
(Rhode, 2009) and teachers (Su, Bonk, Magjuka, Liu & Lee, 2005). Online interactions have been 
associated with increased satisfaction rates (Chang & Smith, 2008), higher course grades 
(Zimmerman, 2012) and decreased course dropouts (Lee & Choi, 2011). However, commercial 
organisations can offer a challenging setting to create interactive online courses. Courses with 
limited opportunities for communication with others (e.g., Padilla Rodriguez & Armellini, 2013a; 
Padilla Rodriguez & Fernandez Cardenas, 2012; Welsh, Wanberg, Brown & Simmering, 2003), 
such as ‘page turners’ and courses consisting mostly of static text, are common. 

The evaluation of course effectiveness poses different challenges. While in academic settings 
learning is usually considered the ultimate goal, in corporate environments, business executives 
tend to prioritise other indicators, such as knowledge transfer (Kirkpatrick & Kirkpatrick, 2010). 
Even so, comprehensive evaluations are uncommon in companies and tend to be limited to a 
satisfaction survey and a final exam (Kim, Bonk & Teng, 2009; Macpherson, Elliot, Harris & 
Homan, 2004; Vaughan & MacVicar, 2004). Sometimes the value of the training is calculated 
using consumption metrics like the number of programmes offered and the duration of completed 
courses (Kirkpatrick & Kirkpatrick, 2010; Macpherson et al., 2004). Some organisations lack 
proper monitoring of training (Kim et al., 2009) and do not even track which employees 
completed which course (Vaughan & MacVicar, 2004). 

This paper aims to provide an understanding of how people interact with content, teachers and 
peers in online courses, a comprehensive evaluation of course effectiveness, and the identification 
of successful course designs in a corporate setting. It addresses the above challenges by reporting 
on a research study focused on the experience of a large commercial food organisation (+6000 
employees) with 30 distribution centres and offices in Mexico. In 2012, this company was 
interested in improving its offer of educational opportunities for its staff by taking advantage of 
their technological resources to deliver effective, interactive online courses. 


Literature Review 

The interaction equivalency theorem (Anderson, 2003) provides guidelines for effective online 
course design through its two theses: 1) Meaningful learning can be supported as long as one of 
three types of interactions (learner-content, learner-teacher or learner-learner) is present at a 
high level. The other two forms can be offered in a minimal degree, or not at all, without 
decreasing the quality of learning. 2) High levels of more than one type of interaction are likely to 
provide more satisfying educational experiences. However, the cost and design time requirements 
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of these experiences might make them less efficient. This theorem is attractive to organisations 
wishing to expand and improve their offering of online programmes, as it addresses limitations in 
social interactions (i.e., between people) by suggesting that meaningful learning will occur if one 
of the other types of interaction can be maximised (Rhode, 2009). 

Online learners tend to reject the idea of different types of interaction being equivalent or 
interchangeable (Padilla Rodriguez & Armellini, 2013a; Rhode, 2009). Nonetheless, perceptions 
may be different from actual behaviours and outcomes (e.g., Caliskan, 2009; Picciano, 2002). In a 
meta-analysis encompassing 74 studies, Bernard et al. (2009) reported that all three types of 
interactions are important for students’ academic achievement. The presence of interactions at 
high and moderate levels was preferable. This finding is consistent with the notion that a high 
level of at least one type of interaction supports meaningful learning. 

Research comparing online course designs emphasising different types of interactions provides 
further insights. Four groups participated in one of such studies (Russell, Kleiman, Carey & 
Douglas, 2009). Group 1 had a high level of social interactions (i.e., learner-teacher and learner- 
learner). Group 2 focused on learner-learner interactions. Group 3 had a teacher but no 
embedded means for communications between students. Learner-teacher interactions happened 
via email. Group 4 had a high level of learner-content interactions, no discussion forums and 
minimum human support available. As Anderson (2003) predicted, outcomes were comparable 
across all four course variations. Participants rated the quality of all courses highly and achieved 
the expected objectives. A similar finding was reported by another study (Tomkin & Charlevoix, 
2014), in which participants of a massive open online course were divided into two groups, one 
with a high level of learner-teacher interactions and one without (but with high levels of learner- 
learner interactions). Completion and participation rates were similar across both groups, as well 
as students’ perceptions on the course. 

A similar study was conducted in a large organisation (Padilla Rodriguez & Armellini, 2014). A 
group of employees experienced three different online courses, each emphasising a different type 
of interaction (learner-content, learner-teacher or learner-learner). Participants in all courses 
achieved the intended learning outcomes. They also reported being satisfied, feeling ready to 
apply the acquired knowledge in their workplace and having their expectations met. Nonetheless, 
findings were mostly based on learners’ perceptions, which indicate that the relationship between 
designed interactions and course effectiveness is not as straight-forward as the interaction 
equivalency theorem (Anderson, 2003) suggests. This article presents the results of a subsequent 
study, which aimed to explore learners’ interactions and incorporate other indicators of course 
effectiveness, following Kirkpatrick’s (1979) guidelines. 

The four-level model of training effectiveness (Kirkpatrick 1979) is the most widely used one in 
corporate online learning (DeRouin, Fritzsche & Salas, 2005; Peak & Berge, 2006). It evaluates 
course effectiveness in terms of (1) reactions (satisfaction), (2) learning (acquisition of knowledge 
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and skills), (3) behaviours (performance in the workplace, or knowledge transfer) and (4) 
business results (organisational level outcomes, such as sales increase). A fifth level (return on 
investment) was subsequently added, and it evolved into return on expectations, or the extent to 
which clients’ expectations are met (Kirkpatrick & Kirkpatrick, 2010). To obtain a benchmark for 
comparison, Kirkpatrick (1979) suggests using either a control group or a pre-post approach when 
conducting evaluations. 

Reports of online courses at organisations tend to be positive. Employees usually express positive 
reactions towards online courses (e.g., Gunawardena, Linder-VanBerschot, LaPointe & Rao, 
2010; Joo, Kim & Park, 2011) and believe that online learning contributes to their personal 
development (Skillsoft, 2004; Vaughan & MacVicar, 2004). Learning outcomes are usually met 
(see review by DeRouin et al., 2005), although sometimes students just want to find specific 
information in the course and leave without completing it once they have learned what they need 
(Skillsoft, 2004; Welsh et al., 2003). While there is limited information about the application of 
levels three and four of Kirkpatrick’s (1979) model, some studies hint that online courses are 
useful to help improve job performance (e.g., Korhonen & Lammintakanen, 2005) and achieve 
business results (DeRouin et al., 2005). 

Could the interaction equivalency theorem be supported when incorporating all of these 
indicators of course effectiveness? This paper addresses this issue. Specifically, data collection 
and analysis were guided by the following research questions: 

1. How do learners interact with the content, the teacher and peers in online courses? 

2. How effective -in terms of satisfaction, learning, knowledge transfer, business results and 

return on expectations- are online courses characterised by high levels of different types 
of interactions? 

3. Which online course design results in the highest levels of effectiveness? 


Method 

Since 2004, course designers at the participating organisation had developed a face-to-face 
Leadership Programme, composed of eight five-hour courses, or modules. Four of these courses 
were selected for redesign and online delivery via the Moodle learning platform. The outcomes of 
the first three are reported in Padilla Rodriguez & Armellini (2014). The fourth one forms the 
basis of this study. 
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Online Course Versions 

A pre-existing face-to-face course on Feedback on Performance was redesigned into three 
versions, each emphasising a different type of interaction (learner-content, learner-teacher or 
learner-learner). Planned outcomes matched Kirkpatrick’s (1979) levels of evaluation. 
Participants were expected to experience an enjoyable, interactive, practical online course 
(reactions), acquire theoretical knowledge on how to provide feedback on performance (learning), 
and apply this knowledge in their job with the members of their teams (behaviours). Students 
who completed the four online courses of the Leadership Programme were expected to increase 
the coverage of their monthly sales quota (business results). 

All versions of the course required approximately five study hours. Their design incorporated ten 
reading texts, five non-assessed activities, a non-assessed final project focused on the application 
of knowledge in the workplace and a final exam. It also included a general discussion forum 
available as an open space for questions and comments, an ethics statement about the study, the 
objectives and structure of the course, and diagnostic and evaluation surveys. To test the first 
thesis of the interaction equivalency theorem (Anderson, 2003) in each version of the online 
course, the researchers attempted to design high levels of only one type of interaction and low 
levels of the rest. Course designers at the organisation validated all materials and activities. A 
brief description of each course version is next. 

Version 1 : Learner-Content (LC) interactions. 

The design and development of this version of the course required a major time investment, as 
content needed to be self-explanatory to compensate for the lack of social interactions. The 
researchers tried to think of all the possible questions students could ask when navigating 
through the course, and to provide answers. Reading texts fostered learner-content interactions 
by including hyperlinks, images and embedded podcasts. Activities required explicit, observable 
responses from the learners. Resources included multiple choice questions with automated 
feedback; a poll that allowed students to see the responses of the group; three podcasts with their 
transcripts; and a personal wiki, which served only as a space for students to write their 
reflections (blogs were blocked). 

Besides the general discussion forum, there were no other embedded communication tools. The 
role of teachers was to monitor student progress without directly intervening. If required, 
teachers could use the general discussion forum to answer questions and clarity tasks. 

Version 2 : Learner-Teacher (LT) interactions. 

Online activities in this course version followed Salmon’s (2002) e-tivity framework, which 
promotes a conversation between participants, and included examples of expected responses. To 
maintain the focus on learner-teacher interactions and to prevent inadvertently fostering peer 
exchanges, activity instructions referred specifically to teacher feedback. Sometimes teachers 
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were asked to reply to each student, and at other times they would address the whole group by 
summarising contributions. Teachers were expected to be an active part of the course, 
maintaining contact, having a regular presence in online discussions (Dennen, Darabi & Smith, 
2007), moderating online learning and providing guidance. Online tools included five discussion 
forums and a wiki per student, available only to the teacher and the learner. Reading materials 
did not include multimedia, as this could increase the number of learner-content interactions. 

Version 3 : Learner-Learner (LL) interactions. 

Activities in this course version also followed Salmon’s (2002) e-tivity framework, which requires 
learners to comment on the work of others (thus generating interactive loops). All five activities 
included a discussion forum and examples. Participants were expected to post at least two 
messages under each activity, one with their solution to the task and a second one replying to 
others’ contributions. No multimedia content was available in this version of the course. Teachers 
were advised to moderate ‘by exception’: Teachers were asked to stand back and let students 
interact among themselves. 

Participants 

Sales supervisors (n=i46, 28 women and 118 men) participated as students of the Feedback on 
Performance course. Their ages ranged from 25 to 57 with a mean of 38 years. On average, they 
had worked at the organisation for five years. Most of them (=62%) had some university studies. 
Others (=31%) had only completed high school (9-12 years of formal education). Few (~ 7 %) had 
only secondary education (6-9 years). 

In their daily jobs, sales supervisors were usually out in the field, visiting supermarkets and 
convenience stores, negotiating sales and talking to retailers in their work team. Most of them 
were not used to office work. Nine months before the study, sales supervisors had received a 
netbook computer with internet access. They had had weekly compulsory training to learn the 
basics of using this technology and how to learn online effectively. They were also enrolled in 
three previous online courses, all part of the Leadership Programme, each emphasising a different 
type of interaction (see Padilla Rodriguez & Armellini, 2014). 

Learners were divided into 18 groups of between 5 and 16 participants (median = 8). The 
distribution was decided by the organisation, on the basis that these groups had worked together 
effectively in the past and should remain unchanged. The average student:teacher ratio was 5:1. 
Sales managers and directors (n=30, 2 women and 28 men) participated as teachers. The 
participating organisation selected them for this role mainly due to their experience in and 
knowledge of the topic. 

Teachers’ age ranged from 27 to 55 with a mean of 41 years. Their average tenure with the 
organisation was six years. All but two of them had at least some university studies. They had 
taken part in training on how to perform as face-to-face and online teachers, and had experience 
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in this role. They received a manual with guidance on how to act according to the course version 
they were teaching and specific examples of expected behaviours. 

Three education staff members performed as academic assistants to the courses, monitoring 
activities and providing general support for participants. Table 1 summarises the participants in 
each version of the Feedback on Performance course. 

Table 1 

Participants in Each Version of the Course 


Course Version 

Groups 

Learners 

Dropouts 

Teachers 

Academic Assistants 

Learner-content 

6 

47 

4 

9 

1 

Learner-teacher 

6 

43 

10 

11 

1 

Learner-learner 

6 

56 

8 

10 

1 

Total 

18 

146 

22 

30 

3 


Human resources (HR) personnel from offices around Mexico participated by conducting 
observations of students. Administrative staff from the sales department provided access to sales 
records. Staff of the systems department (IT services) offered technical support. 

Data Collection and Analysis 

Data collection included several steps that incorporated Kirkpatrick’s (1979) suggestions of a pre¬ 
post approach. Figure 1 summarises this process. 


Baseline 

observations 

Think 

aloud 

sessions 


Exams 


Activity 

logs 


Review 
of sales 
records 

• • 

KM 

BE 

El 

K1 

MSI 

• 

• 

Diagnostic 

surveys 


Review of 
discussion 
forums 


Evaluation 

surveys 


Observations 



Figure 1. Timeline of data collection. 


Before the start of the Leadership Programme, HR staff located in the different offices around 
Mexico attended regular monthly sales meetings and observed the performance of sales 
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supervisors running the meetings (49 students and 14 members of a control group). They used a 
guide created by the researchers to evaluate feedback-related behaviours using a 10-point scale. 

At the beginning of each course version, a diagnostic survey was available online. It consisted 
of closed questions that gauged students’ initial course expectations, previous knowledge of the 
topic and their perceived competence when providing feedback to their team members. Students’ 
self-assessment of their own knowledge and performance served as a reference point for 
comparison with learning and behaviors after the course. 

Students had one week to review the resources and complete all the activities. A convenience 
sample of eight students enrolled in the course version emphasising learner-content interactions 
participated in individual think aloud sessions. This method consisted of observing 
participants, while they verbally articulated their behaviours, feelings and thoughts as they 
engaged with their online course. Throughout the session, the researcher’s input was minimum, 
generally limited to prompts to keep talking when participants fell quiet (Young, 2005). Data 
were audio recorded and transcribed. Transcripts were coded and analysed using NVivo, a 
qualitative data analysis computer software package. Themes for categorisation were based on 
learning strategies students used when interacting with the content. 

To obtain an insight into social interactions (learner-teacher and learner-learner), after the end of 
the course, the researchers navigated through the discussion forums, aiming to identify trends, 
salient features and unusual behaviours, keeping notes of findings. 

A final exam with ten multiple-choice, matching and true-false questions evaluated knowledge 
acquisition at the end of the course. Five items related to a case that students had to analyse. After 
answering, participants could check their grades and feedback. A slightly different version of this 
exam, which included a brief information consent form and could be answered anonymously, was 
available for ten members of a control group, who were not enrolled in the course. Average grades 
for participants in each course version were calculated. 

An evaluation survey included closed questions (5-point Likert scales) about perceived 
engagement with the activities, interactions with content, teacher and peers, and students’ 
evaluation of the course in terms of satisfaction, learning, workplace behaviors and fulfilment of 
expectations. An additional question asked if they had used the private messaging system of the 
online learning platform. Frequencies and percentages were calculated. Open questions in the 
same survey explored learners’ favourite aspect of the course and their suggestions to improve the 
course. Coding focused on references to the different types of interactions. 

Moodle log entries provided additional information about all types of interactions within the 
course. Each log entry contained an action and an information field, which indicated that a click 
happened and specified what the user did. Entries were categorised as passive or active. Viewing a 
resource (e.g., a discussion forum, a wiki or a reading text) was considered passive. Views of the 
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front (landing) page of the course were excluded. Active contributions were clicks that resulted in 
an observable response (e.g., adding a message, editing a wiki or selecting a poll answer). 
Medians were obtained, and ranges (low, medium and high) were determined. The medium range 
was the one that included 50% of participants. 

Learners were characterised in terms of their online behaviours as measured by the number and 
type of clicks (Figure 2). Task-focused students are those for whom finishing tasks takes priority 
over reading support materials. Contributors are those typically considered ‘good students’ who 
review resources, complete activities and share outputs. Average learners are those who fulfil 
requirements to a minimum acceptable level. Observers spend more time looking at resources 
and activities than responding to them. Sometimes they are referred to as lurkers (Salmon, 2011). 
Finally, disengaged students are those who participate little or not at all. The percentage of each 
type of student was calculated. 


(/) 

U 

U 

Q) 

> 

High 

Task-focused 

Contributor 

Contributor 

Medium 

Average 

Average 

Observer 

4-» 

u 

< 

Low 

Disengaged 

Average 

Observer 


Low Medium High 

Passive Clicks 

Figure 2. Student interaction style. 


After the end of the Leadership Programme, new observations were conducted at the monthly 
sales meetings. Average grading of pre and post observations were compared. Data were 
triangulated with students’ perceptions of their own improvement. 

The organisation facilitated access to sales records from 2012, before and after the Leadership 
Programme. These documents provided information about business results (level four in 
Kirkpatrick’s effectiveness evaluation model, 1979). Twenty-two students dropped out of the 
Leadership Programme at different stages or failed to complete it within the allocated time. When 
available, their sales records were taken on board. The average sales quota fulfilment three 
months before and three months after the programme were calculated. Results for sales 
supervisors who successfully completed the Leadership Programme (n=ioi) and those who did 
not (n=38) were compared. A Kolmogorov Smirnov test was used to check if differences between 
groups (i.e., those who completed the Leadership Programme versus those who did not) were 
statistically significant. 

Table 2 matches data sources and the sample for each group. 
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Table 2 

Participants per Data Source 


Data Source 

Learner- 

Content 

(n=47) 

Learner- 

Teacher 

( n =43) 

Learner- 

Learner 

(11=56) 

Control 

Group 

Diagnostic surveys 

46 

3i 

48 

- 

Think aloud sessions 

8 

0 

0 

- 

Exams 

43 

33 

48 

10 

Evaluation surveys 

40 

31 

46 

- 

Observations 

17 

13 

19 

14 

Activity logs 

47 

43 

56 


Sales records 

36 

27 

38 

38 


Results and Discussion 

Results are grouped in terms of the topics of the research questions: participants’ interactions 
within the course, course effectiveness and the comparison between course designs. 

Interactions within the Online Courses 

Students were asked whether they had used Moodle’s private messaging system. Most of them 
had sent at least one private message to another participant (LC: 14/18, LT: 8/14, LL: 10/12). 
They had not been taught how to access the private messaging system. Learners of all groups 
tended to rate peer and teacher participation favourably, by expressing their agreement with the 
following statements: 

•In this course, I have shared valuable learning experiences with other students. 

•The teacher helped me understand the topics of the course. 

Engagement with the courses varied. In the versions with predominantly social interactions, 
online communications faced several problems, such as untimely answers. Students tended to 
become observers, checking discussion forums regularly. In the course emphasising learner- 
content interactions, more participants were contributors and task-focused (Table 3). This finding 
highlights the importance of delivery, which can influence the quality of interactions. The 
following sections detail the interactions in each course version. 
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Table 3 

Distribution of Learners According to their Interaction Style 


Interaction style 

Learner-Content 

Learner-Teacher 

Learner-Learner 

Total 

Contributor 

12 

6 

5 

23 

Task-focused 

3 

0 

0 

3 

Observer 

0 

10 

30 

40 

Average 

21 

15 

13 

49 

Disengaged 

4 

12 

8 

24 


Version 1 : Learner-Content interactions. 

Students could self-pace throughout the online course, as they did not depend on others’ input to 
move forward. Activity logs provided evidence of engagement with the content. Most participants 
had an average interaction style (21/47) or were contributors (12/47; Table 3). Less than half of 
the students (19/47) checked optional resources such as glossary entries. Most students (35/40; 
88%) reported being engaged or very engaged with the activities. 

Think aloud data revealed that when interacting with the content, students used different 
strategies to make information relevant or personalised. For example: 

•Asking questions to themselves. Example: How often do I do this? [...] I am going to write it 
down... 

•Taking notes. Example: When I am going through the course, I always try to have a piece of 
paper by my side, so I can write down what I feel can be useful for doing the activities. It 
is easier that way. If I have problems, here I have a source of answers. 

•Relating the information to their own context. Example: I have a similar case with a retailer I 
supervise. 

•Paraphrasing. Example: Certainly, I think that when we give feedback, when we ask for 
things in the clearest possible way, we promote good communication. 

Some students read superficially, skimming through the text. However, activities (i.e., devices 
designed to promote learner-content interactions) seemed to encourage them to go back and 
spend time on deeper readings. 

During the think aloud sessions, six out of the eight participants had questions that were not 
answered by the content. At times, they would ask the researcher what to do. On other occasions 
they would read their own notes for answers. According to the activity logs, all but one student 
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(46/47) checked the automated feedback received in at least one activity. Incorrect answers 
encouraged students to read again (e.g., “I think I have an incorrect answer... I am going to read 
again and then I will answer [again]”). 

In the evaluation surveys, 29/40 respondents had no suggestions to improve the course. Three 
people mentioned the importance of having embedded social interactions. Yet, students did not 
use the general discussion forum, which was available for questions and comments. Only six 
participants accessed it during the course. 

Although this version of the course fostered no social interactions, the think aloud sessions 
provided evidence of peer exchanges happening outside the online learning environment. During 
all the sessions, either via phone calls or face-to-face interactions, work colleagues interrupted 
students to discuss job matters, which were directly or indirectly related to the content of the 
course. 

An early analysis of this course version is available in Padilla Rodriguez & Armellini (2013b). 

Version 2: Learner-Teacher interactions. 

Most students (28/31; 90%) reported being engaged or very engaged with the activities. While 
23/31 participants had no suggestions to improve the course, three mentioned the importance of 
social interactions. 

Messages reviewed in the discussion forums revealed that no real communication with the 
teachers happened online. Most teachers (8/11) simply did not participate in the course. They 
appeared to be under the impression that the course would teach itself. Three teachers provided 
feedback but in an untimely manner (after the course was over). The quality of these belated 
comments, however, was generally high: Teachers referred to participants by their names, 
expressed their views concisely, provided examples, questioned, summarised, answered, etc. Yet, 
learners who had a response from the teacher did not engage in further dialogue, which questions 
whether they had benefited from those comments. 

Course activities required students to obtain teachers’ feedback to move forward. Learners 
checked the discussion forums regularly, and some of them (10/43) took the role of observers 
(Table 3). On average, each student viewed the five activity-related discussion forums 53 times 
and sent eight messages in total (ten was the minimum required). Many messages were left 
unanswered. 

Even if task instructions did not require it, some students responded to the lack of support by 
taking the role of the teacher themselves and responding to their peers. Other students responded 
to their peers in a shallow way and often just to agree. Either way, learners moved forward, 
regardless the lack of online teacher participation. 

This work is licensed under a Creative Commons Attribution 4.0 International License. 


309 



Expanding the Interaction Equivalency Theorem 
Padilla Rodriguez and Armellini 


Almost a third of students (12/43) were disengaged and reduced their efforts to the bare 
minimum (e.g., posting shallow messages and failing to complete activities). In the evaluation 
survey at the end of the course, one of the participants requested: “Let there be support from the 
facilitator.” 

Version 3: Learner-Learner interactions. 

Most students (44/46; 96%) reported being engaged or very engaged with the online activities, 
which required students to respond to others and to check others’ comments on their 
contributions. Participants checked the discussion forums regularly. Most students (30/56) took 
the role of observers (Table 3). On average, each student viewed the six activity-related discussion 
forums 83 times and sent 12 messages, which is consistent with the minimum number of posts 
expected. 

Some messages were left unanswered. Other answers were posted too late to benefit others. 
Teachers in this version of the course provided no clarification, although they were expected to: 
Moderation was supposed to occur in cases of confusion. Messages reviewed in the discussion 
forums revealed that no meaningful communications between learners happened online. A high 
number of replies consisted of brief comments agreeing to other participants’ contributions, or 
telegraphic, hard-to-interpret messages, as if students were just responding to fulfil the activities’ 
requirement and tick the box. Learners who had a response from others would rarely or never 
reply. 

Some participants did use the discussion forums to elaborate on the topic (e.g., It may be very 
motivating to acknowledge the efforts of our collaborators, but it is more important the way in 
which we provide the feedback), provide suggestions (e.g., “Your feedback is motivating, but you 
need to be more specific. If you generalise, you might neglect important details”), and show 
support (e.g., “You have the experience and skill to achieve [your goal]. Remember that each 
retailer has a different level of motivation, and you have the way of making them be motivated”). 

Some learners found contributions from others beneficial. In the evaluation surveys, when asked 
about the aspect of the course they had enjoyed most, nine answers (out of 46) referred to 
learner-learner interactions. For example: “The participation of my course mates, which at the 
same time served me as feedback.” 

Course Effectiveness 

Course effectiveness was generally high, in line with studies reporting positive reactions towards 
online courses (Gunawardena et al., 2010; Joo et al., 2011), learning (DeRouin et al., 2005), 
improvement of job performance (Korkhonen & Lammintakanen, 2005) and achievement of 
business results (DeRouin et al., 2005). Despite the issues identified above (i.e., unanswered 
questions, lack of feedback and shallowness of interactions), students managed to meet the 


This work is licensed under a Creative Commons Attribution 4.0 International License. 


310 



Expanding the Interaction Equivalency Theorem 
Padilla Rodriguez and Armellini 


intended course outcomes. Table 4 summarises the findings in terms of each level of 
effectiveness, for each course version. 

Table 4 


Comparison of the Effectiveness of each Course Version 



Learner- 

Content 

(n=47) 

Learner- 

Teacher 

(n=43) 

Learner- 

Learner 

(n=56) 

Control 

Group 

Reactions: 

Percentage of satisfied learners 

97-5 

too 

97-8 


Learning: 

Average exam grades (out of 10) 

9-5 

9-4 

9.6 

7-1 

Behaviours: 

Average observed change 
(percentage points) 

1.9 

-4-7 

1-7 

-2.0 

Results: 

Increase in sales quota coverage 
(percentage points) 

9-4 

4-5 

7-9 

6.7 

Return on expectations: 

Percentage of learners reporting 
fulfilment of expectations 

too 

too 

too 



Note. There were no statistical differences between groups. 


Twenty-two employees dropped out of the courses. Eighteen of them were enrolled in the versions 
emphasising social interactions. Perhaps they felt disengaged due to the mostly shallow and 
untimely exchanges online, or they considered they had learned what they needed before 
completion (Skillsoft, 2004; Welsh et al., 2003). 

Participants’ initial course expectations encompassed all the effectiveness levels defined by 
Kirkpatrick (1979). Most learners (81-88%) expected to acquire and apply knowledge and to 
translate this into increased sales. At the end of the course, regardless of the type of interactions 
emphasised, all participants reported in the evaluation surveys that their expectations had been 
met. These data are in line with reports of other employees who perceive that online learning 
contributes to personal development (Skillsoft, 2004; Vaughan & MacVicar, 2004), and with 
students’ perceptions of being satisfied with their course. Participants in this research claimed 
they had learned and felt prepared to provide effective feedback to their collaborators. 

Comparison between Course Designs 

Considering student satisfaction, learning outcomes and return on expectations, all courses were 
equally effective, regardless of the type of interactions emphasised (Table 4). A Kruskal-Wallis 
test was run to compare the exam grades of the learners who took the different course versions. 
Results were not statistically significant, which is consistent with previous studies (Bernard et al., 
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2009; Padilla Rodriguez & Armellini, 2014; Russell et al., 2009; Tomkin & Charlevoix, 2014). 
They suggest that the interaction equivalency theorem (Anderson, 2003) is applicable in a 
corporate setting. 

There were differences when considering behaviours and business results. Students in the course 
emphasising learner-teacher interactions did worse than those in all other groups (Table 4). Since 
disengaged teachers were also students’ line managers, it is likely that participants received no 
encouragement to apply in the workplace what they had learned in their online courses and 
translate it into business results. Organisational support has been linked to knowledge transfer 
(Gunawardena et ah, 2010; Joo et al., 2011). 

Delivery and online moderation posed new challenges for course effectiveness, which were not 
fully incorporated into the design phase. As described, these problems included learners having 
questions not answered in the course materials, teachers being disengaged and students sending 
shallow contributions. Both design and delivery are important for effectiveness. Figure 3 
exemplifies possible scenarios for online courses emphasising a single type of interaction. 


a: 

LU 

> 


LU 

LO 

cn 

D 

o 

u 



Predominant Interactions 


Learner- 

Learner- 

Learner- 


Content 

Teacher 

Learner 

Q) 

Students need 

Teacher 

Peer feedback 

fO 

3 

an extra effort 

provides 

and support 

O - 

0) 

to make sense 

tangible 

become 

“O 

< 

of the content 

enhancement 

central to 



from base 

course 



materials and 

success 



activities 



High likelihood of achievement of 
course outcomes 


0) 

01 

3 

rx 

01 

"O 

ro 

c 


High risk of course failure 


Students require support alternatives, 
resulting in some disengaged learners 


Inadequate 


Adequate 


COURSE DESIGN 


Figure 3. Scenarios resulting from different types of course design and delivery. 


Note. Adequate course design includes clear, well-structured materials and activities focused on 
promoting meaningful interactions. Adequate course delivery has easy-access content, effective 
technology use, and meaningful social interactions. 
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When both design and delivery are adequate, there is a high likelihood that course outcomes will 
be achieved. When both design delivery are inadequate, the course will probably fail. When there 
is adequate design but delivery is inadequate (as happened in this study in course versions with 
predominantly social interactions), students will require additional support alternatives, which 
they may find within or outside the course boundaries. Some learners will be successful in finding 
them; others will become disengaged. Delivery can provide a tangible enhancement to an online 
course, even when its design is inadequate. 


Conclusions 

The interaction equivalency theorem (Anderson, 2003) applied in the corporate setting of this 
study: Deep, meaningful learning can be supported as long as one of three types of interaction 
(learner-content, learner-teacher or learner-learner) is present at a high level. Results from 
course versions emphasising different types of interactions showed that participants obtained 
similar exam grades and reported similar perceptions about their learning. Thus, this study tested 
and provided empirical support for the theorem in a commercial organisation. 

This study considered indicators of course effectiveness relevant in organisational contexts: 
learner satisfaction, behaviours in the workplace, business results (Kirkpatrick, 1979) and return 
on expectations (Kirkpatrick & Kirkpatrick, 2010). All participants reported feeling satisfied with 
their courses and having their expectations fulfilled. Those students in courses with 
predominantly learner-content or learner-learner interactions also showed improvement in their 
communication skills and increased sales results. In courses emphasising learner-teacher 
interactions, most teachers were disengaged. While students achieved learning outcomes, their 
behaviours in the workplace and business results were not as good as those of participants in 
other groups. The quality of course delivery is crucial for success. 

Findings suggest that the interaction equivalency theorem can be reformulated as follows: In 
corporate settings, an online course can be effective in terms of satisfaction, learning, knowledge 
transfer, business results and return on expectations, as long as it meets the following criteria: (a) 
at least one of three types of interaction (learner-content, learner-teacher or learner-learner) 
features prominently in the design of the course, and (b) course delivery is consistent with the 
chosen type of interaction. 

Criterion (a) refers to what Anderson (2003a) called a ‘high level’ of at least one of the three types 
of interactions. It implies the design of multiple online activities, which require observable 
responses from participants and generate interactions with the content, the teacher or other 
learners. Criterion (b) requires designers and other practitioners to consider course delivery, 
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which should be planned and managed to maximise the benefit and impact of the predominant 
type of interaction designed into the course. 

While designers and educators cannot control participants’ online behaviours or guarantee that 
interactions will be meaningful, they should incorporate sufficient opportunities for exchanges 
and ensure that support channels are available. In this study, learners were resourceful when they 
faced the disadvantages of the interactions designed into their courses. They engaged in informal, 
unplanned learning activities beyond course requirements, on and offline. If interactions 
embedded in their courses did not provide answers to their questions, they looked for 
alternatives, such as reviewing their own notes, communicating privately with others via Moodle 
messages or talking face to face with colleagues. These activities relate to all three types of 
interactions and have a potential impact on course effectiveness. 

Incorporating more than one type of interaction may compensate for the disadvantages of the 
chosen form of embedded interactions. This idea may appear to be similar to the second thesis of 
Anderson’s (2003) interaction equivalency theorem: High levels of more than one of three types 
of interaction (learner-content, learner-teacher and learner-learner) are likely to provide a more 
satisfying educational experience but at a higher monetary and time cost than less interactive 
courses. However, the recommendation is to ensure the course does not rely on a single type of 
interaction, rather than to include high levels of several types. 

This reformulation of the interaction equivalency theorem provides guidelines for the design and 
delivery of effective online courses at organisations. It may be valuable for academics and 
practitioners interested in corporate online learning. 
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